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Abstract 

Evolutionary optimisation algorithm is employed to design networks of phase-repulsive oscillators 
that achieve an anti-phase synchronised state. By introducing the link frustration, the evolutionary 
process is implemented by rewiring the links with probability proportional to their frustration, until 
the final network displaying a unique non-frustrated dynamical state is reached. Resulting networks 
are bipartite and with zero clustering. In addition, the designed non-frustrated anti-phase synchro- 
nised networks display a clear topological scale. This contrasts usually studied cases of networks with 
phase-attractive dynamics, whose performance towards full synchronisation is typically enhanced by 
the presence of a topological hierarchy. 



1 Introduction 

Study of collective phenomena in complex systems has been massively enhanced in the last decade 
by introduction of the complex networks framework [1]. This revolutionised our understanding of the 
real complex systems, ranging from gene and neuron networks, to air traffic, Internet and society [2]. 
This paradigm allowed for novel theoretical and computational investigations, such as stability of gene 
regulatory networks examined through the network-inhibition of chaos [3], or detection of communities 
in social networks [3]. 

The phenomenon of synchronisation occupies a prominent place in the networks science, as a par 
excellence example of emergent collective behaviour [5]. Synchronisation of various kinds of oscillating 
dynamical units have been extensively studied both theoretically [6] and experimentally [3- The emer- 
gence of synchronisation is often examined by employing Kuramoto oscillators fS] . For being conceptually 
simple and easy to implement numerically, Kuramoto oscillators allowed for synchronisation to be ex- 
amined in great detail [5l[6]. This model is often used in theoretical and experimental studies, such as 
network reconstruction [9], or modelling of neural phase-resetting curves [lOj . 

In parallel with synchronisation, research attention was devoted to various forms of anti-synchronisation [llj . 
This particularly refers to anti-phase synchronisation [T2], where the interacting oscillator pairs evolve 
towards having the opposite phases, i.e. towards being synchronised with a fixed phase difference vr. In 
this context, networks of units with the repressive and/or repulsive interactions were investigated. Most 
famously, the system of repressively interacting genes - repressilator - received much attention [13]. Re- 
pulsive interactions can generate dynamical frustration and multistability, since interacting units cannot 
always relax to a unique equilibrium state \1A\ I15j . Neurons are often coupled repulsively, which was 
examined theoretically jl6j and experimentally [T7]. In addition, repulsive Kuramoto oscillators can be 
used to model social interactions and social equilibria [18] , or suppression of synchronisation [19] . 

Recently, complex networks of Kuramoto oscillators will solely repulsive coupling were examined [20] , 
in opposition to studies involving both attractive and repulsive interactions |21j . By borrowing the 
terminology from disordered systems [22], the dynamical states in these networks were characterised by 
defining the link frustration f. In general, complex networks were found to have positive total frustration, 
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due to complexity of their topologies. In addition, depending on the initial conditions, networks typically 
display multiple final dynamical states. The network's emergent dynamics is thus strongly dependent on 
the topology, and for a general network does not reach "minimum energy" state with zero frustration. 

In a different context, researchers have been studying the problem of network design, motivated by the 
technological need to engineer networks that perform prescribed functions. Models are typically based 
on a network whose topology is evolving towards the optimal target topology. Most used among such 
models is the evolutionary optimisation, a version of Monte Carlo simulated annealing algorithm [23]. A 
myriad of results were obtained through this formalism, such as emphasising peculiarities of the scale-free 
topologies [M]! design of biological transduction networks [521 ES], design of robust genetic clocks [57], 
or design of networks that optimally perform a given dynamical behaviour [28] . In the context of oscilla- 
tors, design of easily synchronisable network was recently examined [29], and a method of achieving the 
prescribed synchronisation state by configuring a repulsive subnetwork was exposed |30j . 

A question that remains open in the context of phase-repulsive networks, regards the topologies that 
allow for the zero frustration state to be attained. These are the networks whose dynamics leads to 
anti-phase synchronisation for any choice of initial conditions. While essentially any oscillator network 
with phase-attractive coupling generates full synchrony, obtaining anti-phase synchronisation is more 
demanding. As shown in [20], despite that phase-repulsive interactions naturally tend towards anti- 
phase synchrony, a generic network typically can not attain fully anti-phase synchronised state due to 
the topological constraints. In this paper we employ the evolutionary design algorithm \23\ \25\ [27]. to 
obtain phase-repulsive oscillator networks that always achieve non-frustrated, i.e. anti-phase synchronised 
state. By choosing the links to rewire according to their frustration, the initial network topology evolves 
towards minimising global frustration, which inevitably leads to non-frustrated topology. As we show, 
in opposition to the well studied synchronisable networks [29], whose synchronisation performance is 
typically enhanced by hierarchical or scale-free structures [31], non-frustrated networks are topologically 
very uniform, and exhibit a precise scale |32[ [33] rather than being scale-free. This problem is similar 
to the problem of 2-color vertex colouring encountered in graph theory [3l]. The difference is that the 
process of vertex colouring is here automatically done by the network phase-repulsive dynamics. 

We develop our model by considering a network that consists of oscillators (nodes), connected via 
L non-directed links. Dynamical state of the oscillator i is given by the phase variable y?,, € [0, 27r), and 
its dynamics is defined by: 

(pi=UJi + -j- J2 ^ij9{fj - fi) > (1) 

where ki is the node's degree (J^j ki = 2L), e is the coupling strength, and uji is the oscillator's natural 
frequency. Network's topology is expressed via symmetric adjacency matrix Aij, defined as Aij = 1 if 
nodes i and j are connected, and Aij = otherwise. We consider identical oscillators = w, and set 
g = sin, reducing our system to the simple Kuramoto model [8] . Following the results exposed in [20] , we 
examine the network model with phase-repulsive coupling. To that end we fix e = —1, and for simplicity 
take w = 0. The Eq(T]for our model reads: 

(pi = ^ Aij sin{(pj -ipi) . (2) 

Time-evolution starts from a random set of initial phases (IP), selected independently for each oscillator 
from 93j(0) G [0,27r). The interacting oscillator pairs evolve towards maximising the phase difference 
between them, i.e. their phase values "stretch" apart from each other as much as possible. Preferably, 
the maximal phase difference is tt. However, for a general network, due to its complex topology, the phase 
difference along certain links is often less than vr, or even zero. The final (stationary) dynamical state is 
quantified by introducing frustration fij for each link as |20[ I21j: 

fij = Aij[l + cos{ipj - ipi)] . (3) 

Frustration measures how "squeezed" is a link: it can be pictured as the elastic potential energy contained 
in it: a link stretched to the phase difference vr has zero frustration, whereas a link forced to synchronise 
carries the maximal frustration that equals 2. We characterise the network dynamical states by specifying 
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the link frustration values fij. To measure the global frustration we define F as the network average of 
/: 

i>j 

which quantifies how much does the network topology allow for links to stretch. Note that F can be seen 
as the non-equilibrium potential, since Eql2]can be written as [21j : 

= (5) 

ki dipi 

As concluded in [2D], phase-repulsive dynamics for a general network leads to a positive total frustration, 
whose value strongly depends on the details of network topology. In widely studied phase-attractive 
case, full synchronisation is essentially the only final dynamical state, independently of the topology. In 
contrast, topology in our model crucially determines the dynamics. In addition, a general phase-repulsive 
network displays multiple final frustration states, each occurring for a certain fraction of IP. 

Interestingly, some networks exhibit a unique dynamical state with zero total frustration F = 0. These 
networks are completely anti-phase synchronised - each coupled oscillator pair has the opposite phases 
for any IP. This comes from peculiar topologies of such networks that allow for all links to stretch, thus 
completely avoiding frustration. While in the phase-attractive case, basically any network achieves the 
fully synchronised state, in the phase-repulsive case only a limited set of networks attain the anti-phase 
synchronised state. In addition, such state is also stable, as it is the only possible dynamical equilibrium 
state. How can we construct non-frustrated networks? What are their topological properties? The rest 
of this paper is devoted to these two questions. 



2 Results 

We employ the evolutionary algorithm based on simulated annealing in order to design non-frustrated 
networks. The evolutionary process involves rewiring links with the probability proportional to their 
frustration fij, until a network with zero frustration along all links for all IP is obtained. Further details 
on our network design method are provided in Methods section below. 

An example of a simple evolutionarily designed non-frustrated network with = 20 nodes and L = 30 
links is shown in Fig(TJ It was obtained after 15 evolution steps. The network appears to be uniform 




Figure 1: Example of an evolutionarily designed non- frustrated network, with = 20 nodes and L = 30 
links. All adjacent node pairs are having the opposite phase values (phase difference vr), and hence zero 
frustration, for any choice of IP. Nodes appear in two shades in order to show bipartitness. Each node 
has zero clustering. 

in structure, without prominent hubs. It is also bipartite, as illustrated by different shades of adjacent 
nodes. Large presence of 4- node rings, which are small non- frustrated networks, is clearly visible. There 
are also 6-node rings, which are unable to exhibit the higher F state due to being topologically intertwined 
with the rest of the network, which is forcing them into F = state. Clustering coefficient is zero for all 
nodes. Due to a large number of 4-node rings, the network topology seems more regular than expected 
from an evolution governed by random rewirings. 
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To quantify more precisely the topological properties of non- frustrated networks, we now consider 
larger networks with N = 100 nodes. We run evolution simulations with various numbers of links, 
ranging from L = 200 to L = 2000: all evolution processes yield a final non-frustrated network. In 
particular, we here focus on networks with L = 200, 400, 600 and 800, which are shown in FiglJl and 
indicated via the corresponding L (details on this evolution process are included in the Methods section 
below). As observed earlier with Fig{Tl the non-frustrated networks seem not to posses any hierarchical 




Figure 2: Four networks with zero total frustration, obtained through evolutionary processes described in 
the Methods section, with N = 100 and L as indicated. Again, networks attain anti-phase synchronisation 
for any choice of IP, are bipartite and have zero clustering. Networks of size beyond 800 are discussed in 
the Supplement figure. 

structure, which is even more apparent here. Of course, all the obtained networks are bipartite and have 
zero clustering. We run many evolution simulations with the same and L: the same global topological 
features were always observed. As we show more quantitatively in what follows, the structures of these 
networks are indeed surprisingly regular. 

We first compute the histograms of node degrees. To this end, we average the degree histograms over 
5 realisations of the non-frustrated networks with given N and L, and report the results in Figj3j All 
histograms are very localised in range and display a well pronounced peak in the centre, whose value 
coincides with the average degree (ki) = 2-^. As expected, for the increasing network size, the peak 
becomes gradually less pronounced. This confirms that our networks are not hierarchical. In order to 
nullify the frustration, network evolution leads towards evenly spreading the remaining link frustration 
over the network. This indicates that rather than being scale-free, non-frustrated networks are "scale- 
rich", i.e. posses a well defined scale [52] (this holds for small L, i.e. sparse networks). The scale is 
determined by the average node degree. 

The node degree histograms suggest that non-frustrated networks posses structures that are similar 
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Figure 3: Histograms of node degrees for four non-frustrated networks with N = 100 and L as indicated, 
in correspondence with Figl2j Data is averaged over 5 reahsations of such networks. Each histogram 
displays a prominent peak, coinciding with the average node degree for each network. 

to Erdos-Renyi random networks, that the evolutionary process started from. However, two types of 
structure also have profound differences. Non-frustrated networks, by being bipartite, always have zero 
clustering. In contrast, the average expected clustering coefficient for random networks is [1]. This 
difference is a clearly pronounced in dense non- frustrated networks with high {ki). In addition, we found 
that non-frustrated networks systematically display somewhat longer average shortest paths than random 
networks with the same N and L. This can again be attributed to zero clustering: since no three nodes 
can be mutually interconnected, shortest paths are on average somewhat longer. Regardless of N and L, 
average shortest path in any non- frustrated network can never exceed |, since on average, for each node, 
a half of the remaining nodes are at least two links away. 

There is an upper bound on L for any given N , beyond which it is not possible to design non-frustrated 
networks, since avoiding connected node triangles becomes impossible. Close to this limit the network 
evolution process gradually becomes slower. The network structure however becomes even more regular, 
since avoiding frustration requires more and more ordered organisation of links, and more uniform node 
degrees. In the supplementary figure to this paper, we show the additional six networks with = 100, 
and L ranging from 1000 to 2000. The regularity and bipartitness of these dense networks are clearly 
visible. 

Finally, we quantity the topological regularity of non- frustrated networks. We achieve this by mea- 
suring the concentration of motifs in our networks, using the mfinder software j35j . We first introduce 
Zgcore, as a measure of how over- or under- represented is a given motif w within some network. It is 
defined as [36] : 

Zscoreiw) = , (6) 

where Nactuai{w) is the number of motifs w found in the examined network, Nj.andom{w) is the average 
expected number of those motifs in a network with the same and L, while SD{w) is the standard 
deviation of Nrandom{w). Thus, any motif whose Z score is considerably bigger that 1 (-1) is over- (under-) 
represented. Since the studied networks differ in their numbers of links, rather them employing Zgcore, 
we instead rely on normalised Z score-, which is defined as [26]: 

Znormalisediw^ = / == . (7) 

V l^w Ziscore\W) 

We calculate Znormaiised for all 4- node non-directed motifs (shown in FiglSj:), for four evolutionarily 
designed networks shown in Figl21 The results are reported in FigH] In all networks, non-frustrated 
motifs are overwhelmingly over-represented, while the frustrated motifs are drastically under-represented 
(cf. FigU]). Both over- and under- representation are invariant to varying realisations of non-frustrated 
networks, and to the number of links L. The same result is found for both 3-node motifs: the chain 
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Figure 4: The value of Znormaiised for all non-directed 4-node motifs in the networks considered in Figl2] 
as indicated, computed using mfinder software |35j . Non-frustrated motifs are over-represented, while 
the frustrated motifs are under-represented. Profiles are invariant to the network size. 

is systematically over-represented, while the ring is strongly under-represented. This suggests that, 
regardless of their size, non-frustrated networks have an inherent motif pattern, rooted in their dynamical 
properties. This result confirms that large non-frustrated networks are essentially composed of small 
non-frustrated motifs. By locally forming non-frustrated motifs, mechanism of network evolution evolves 
towards global zero frustration. Interestingly, this means that the emergence of global = is achieved 
locally, by each network building block evolving towards F = 0. This implies that any large non- 
frustrated network can be decomposed into smaller non-frustrated subnetworks (an alternative scenario 
can be imagined, where the desired global property is achieved only globally, i.e. only at the level of the 
entire network, with no subnetworks exhibiting the same property). 

3 Discussion 

Natural evolutionary processes in real complex networks often lead to scale-free structures in the emerging 
networks |24] . However, the ubiquity of scale- free topologies has been recently challenged. New findings 
include examples of real complex networks, that despite emerging from biological self-organisation, posses 
a clear scale in their structure, contrasting widely accepted universality of scale-free topologies |32tl33j. In 
this paper, we showed a model of network topology that develops under a simple evolutionary constraint, 
whose topology evolves away from hierarchical structure, and towards a "scale-rich" structure [32]. As 
shown above, this can be attributed to the nature of phase-repulsive dynamics, which requires certain 
topological properties in order to attain fully anti-phase synchronised state (i.e., non- frustrated state). 
Hence, in opposition to the phase-attractive dynamics and resulting full synchronisation, that are typically 
improved by the presence of a topological hierarchy, anti-phase synchronisation appears to be enhanced 
by the lack of hierarchy. 

Bipartite networks are usually considered in relation to two different types of nodes, such as on-line 
comments and the corresponding web-page users (persons) [57]. Here we showed that bipartitness can 
spontaneously emerge from the evolution of a network with a single type of nodes (identical oscillators). 
Besides these findings, this work confirmed again that evolutionary design through random link rewirings 
is a viable method of constructing complex networks that exhibit prescribed dynamical patterns. Our 
method mimics the natural evolutionary processes, by implementing random mutations (rewirings). To 
make mutations completely random, one could modify the algorithm to choose links for rewiring regardless 
of their frustration. This would however only slow down the process, without changing the final result. 
On the other hand, it is possible that the non-frustrated networks can be obtained by means of other 
design methods. Such methods are however still poorly explored, even in the context of simple models 
such as oscillator networks. 

We conclude this section by examining the limitations and extensions of our results. As noted ear- 
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lier, zero frustration is not achievable beyond a certain limit of L. Our algorithm can in this case be 
applied to minimising the frustration, and studying the ways in which the resulting networks organise 
the remaining frustrated links. Another question refers to maximising instead of minimising the total 
network frustration. By accepting the mutations that increase rather than decrease F, our algorithm can 
be used to design networks with maximal obtainable frustration. Intriguing problem here is the differ- 
ence between the frustrated (squeezed) and non-frustrated (relaxed) network topologies. This is related 
to elastic network models, where mechanical properties of networks of elastic springs are studied |38] . 
It would be also interesting to apply the same algorithm to design non-frustrated networks using other 
models of repulsive interactions. This particularly refers to neural and gene interactions, where networks 
with repressive interactions are widely studied \14:\ \T5\ [16] . This could aid engineering genetic or neural 
networks with a given prescribed functionality. Finally, we note that our networks were designed to 
satisfy a simple dynamical rule, formulated as F = 0. By varying the dynamical rules, one could use this 
or similar algorithms to design networks whose topological properties reflect a predetermined dynamical 
rule. This could lead to the discovery of new complex network topologies, that do not fit into any of the 
currently studied topological categories. 



4 Methods 

In this section we describe our method of evolutionarily network design and explain its implementation. 



Preliminaries. For small networks it is easy to find non-frustrated topologies, as shown in FiglSl A con- 
nected pair of nodes, 3- node chain, 4-node star, 4-node chain and 4-node ring (cycle), are networks that 
show a single frustration state F = 0. However, finding a large non-frustrated network of size ^ 1, is 
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Figure 5: Frustration for two-node network in (a), both three-node networks in (b), and all six possible 
four-node networks in (c). Total frustration F is reported, along with all link frustrations fij G [0,2], 
illustrated by the (color)scale. Frustration is defined in Eql3]and EqU and refers to the dynamical model 
EqiJl Note that three four-node networks are frustrated, and the other three are not. 

not straightforward. A typical network will in general have F > 0, for instance, due to having at least one 
3-node ring. On the other hand, it is easy to notice that for a network to be non-frustrated, it needs to be 
bipartite, and hence have zero clustering. Indeed, for having two groups of nodes with opposite phases, 
the network needs to involve links that exclusively go from nodes of one group to the nodes of the other 
group. However, although necessary for anti-phase synchronisation, bipartitness is not sufficient for it. 
In FigEJwe show the 6-node ring, which is a bipartite network. However, besides displaying the expected 
non-frustrated state, the network possesses another frustrated state with F = 0.5, that occurs for 14% 
of IP. Bipartitness alone guarantees the existence of a non- frustrated state, but it does not guarantee 
its uniqueness. This indicates that we need a more elaborated mechanism for designing non-frustrated 
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Figure 6: Two frustration states exhibited by the 6-ring. Total frustration F and the hnk frustrations 
fij € [0, 2] are shown for each state. The non-frustrated and the frustrated states appear for 86% and 
14% of IP, respectively. 

networks with the unique F = state. 

Evolutionary algorithm. We employ the simulated annealing algorithm as follows. Starting from a 
random network, at each evolution step we rewire one of the frustrated links, and accept the mutation if 
the new network exhibits smaller total frustration F than the original one. The network topology evolves 
until the target F = is reached. More precisely, our algorithm goes as follows: 

0. Start from a non-directed, connected network TV, with nodes and L links (A^ — ^ < L < ^(^~^) 

and no self-loops. We use Erdos-Renyi random network. Compute link frustration values fij and 
total frustration F, averaged over M random IP. We use M = 5. 

1. Select a link ij in J\f with probability proportional to fij + a (we use a = 0.01). Detach this link 

from one of its end-nodes (randomly chosen), and rewire it to a different node chosen with uniform 
probability, making sure that the network stays connected and without self-loops. This is the 
mutated network J\f'. 

2. For M' compute flj and F'. If F' < F the mutation is accepted, and the network is updated: M = N' . 

li F' > F the mutation is accepted with probability ^-{F' ^ Otherwise, the mutation is 
rejected and TV remains unchanged. We use a = 0.02. 

3. The process continues from step 1 by selecting a new link in TV to be rewired. 

The evolution continues until the final network tV^ with a unique frustration state F = is reached. 
Potential multiple frustration states are excluded by averaging over M random IP. The numerical con- 
stant a skews the probability distribution allowing for non-frustrated links to be mutated as well, while 
ensuring that more frustrated links are still mutated more often. This drastically expedites the evolution 
process, while at the same time allowing for all links to be mutated. Parameter a allows that, with a 
small probability, even mutations leading to a small increase of total frustration get accepted. By playing 
the role of temperature, a prevents the evolution process from getting stuck into local minima. 

Algorithm's performance. We tested the evolution algorithm exposed above on numerous network 
examples: the process invariably leads to a non-frustrated network. Since the evolution involves random- 
ness, the final network does not depend on the starting one. On the other hand, for any fixed and 
L there are many different realisations of final non-frustrated networks, all sharing the key topological 
properties. To illustrate the progress of the network evolution, we show in Figi7]the gradual decrease of F 
for four examples of evolution simulations related to networks from Figl2j The evolution speed decreases 
with the increase of L. In a denser network it is harder to find a mutation that leads to a decrease of F, 
and when found, that mutation typically decreases F for a small amount. In addition, denser networks 
start with a larger initial F, thus additionally increasing the needed number of evolution steps. We 
adjusted for the optimality of the process through the parameters M, a and a. The "bumpiness" of the 
processes is due to Monte Carlo algorithm's temperature o" > 0. This parameter needs to be adequately 
set to optimise the evolution speed and effectiveness. Too small a might result in process getting suck 
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Figure 7: Network evolution processes generating four networks considered in Figl2j The value of total 
frustration averaged over M = 5 IP -F is shown, as function of the number of evolution steps. 



into a local minimum of F, which may incorrectly suggest that smaller F is not obtainable. Too large 
a could induce the process to wander too excessively, without ever finding F = solution. Our choice 
of cr = 0.02 takes into account both issues. At each evolutionary iteration, the network usually has 
multiple frustration states with different F-values. To obtain a single frustration value F corresponding 
to the current network, we average F-values over M = 5 IP. This is an additional reason for introducing 
temperature a: due to a potentially wide range of multiple states, averaging might not give the best 
measure of the total frustration. Upon each completion of evolutionary simulation, we checked that the 
obtained network is indeed non- frustrated. 
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Figure Legends: 

Figure 1 Simple example of an evolutionarily designed non-frustrated network. 
Figure 2 Large evolutionarily designed non-frustrated networks. 
Figure 3 Histograms of the node degrees in non-frustrated networks. 
Figure 4 Motif concentrations in non-frustrated networks. 
Figure 5 Frustration in small networks. 

Figure 6 Two frustration states exhibited by the 6-ring network. 
Figure 7 Illustration of the network evolution processes. 
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